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REMARKS 

This responds to the Office Action dated December 4, 2006. 

In the Office Action, claims 1-44 are noted as pending in the application, claims 
1-44 stand rejected, no claims are objected to and no claims are allowed. No claims 
have been withdrawn from consideration. 

Status of the Claims 

Claims 1-44 are pending in the application. New claims 45-57 have been added 
and claim 7 is canceled without prejudice. 

Rejections 

The Office Action rejects all pending claims 1-44 and discusses each claim 
separately, which Applicant appreciates. Each rejection of the Office Action also cites 
to particular portions of the cited reference under consideration. However, not all of the 
citations to the references relate to the particular rejection for which the citation is made, 
which means that no support is given in the Office Action for the rejection. Additionally, 
as to those rejections, the undersigned representative has been unable to locate in the 
cited reference after reasonable review any support for the particular rejection. For 
example, the rejection of claim 6 recites "Al-Hussein discloses the method (fig 1-5), 
wherein the step of characterizing includes the step of identifying pixels representing 
color information; see (col. 18, lines 1-15)." However, the citation says nothing about 
color, and upon review, nothing elsewhere in the cited reference supports the rejection. 
The rejections which lack any support because the citations are erroneous are the 
rejections for at least claims 6, 7, 18, 19, 21, 35, 37, 39, 40, 41, 42, and 43. Therefore, 
the rejections of these claims fail without more, and the rejections of these claims 
should be withdrawn. While one or more other arguments herein may also apply to 
overcome these rejections, these rejections should be withdrawn without more because 
the Office Action fails to support the rejections. 

In the Office Action, the claims are rejected as being allegedly obvious over Al- 
Hussein (USP 5,818,978) in view of Luther et al. (USP 6,449,065). Necessarily, neither 
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of the references alone teach or suggest the claimed combinations, and the rejections 
must necessarily rely on at least two references for teaching or suggesting the claimed 
combinations. In the present application, however, the combination of the references as 
argued in Office Action does not produce the claimed inventions, as demonstrated 
below. 

Applicant's Disclosure 

Consider first Applicant's disclosure. Applicant discloses methods and apparatus 
are described for analyzing images, for example to permit image segmentation, 
recognition of image interior portions, improved image enhancement, and/or to permit 
data compression or other processes. One or more of the methods permits easier 
image characterization, for example at multiple levels, allows disassembly of image 
layers or segments, takes into account a number of color attributes, can make easier 
such image enhancements as edge sharpening, selective and uniform image 
enhancement, color and black and white enhancement or enhancement of color text. 
More precise image rendering is possible, and image enhancement based on image 
content can be improved. Image processing can also be improved, for example, by 
removing or modifying color anomalies that may have been generated in an otherwise 
black and white, text document, thereby making printing of the black and white 
document on an inkjet printer more efficient by reducing the use of color ink cartridges. 
In one application of one of the present inventions, image characterization can be used 
to identify an image such as one from a scanner as one of either all text, all graphic or 
picture, or a mixture of both text and graphic. Such information can then be used to 
establish scanner settings, parameters used in image enhancement, as well as for other 
purposes. 

Considering the image analysis and characterization step in more detail, in one 
form of a process for analyzing an image and characterizing the image as text only, 
picture or graphic only, or mixed, image data is received and elements of the image are 
characterized, preferably on a pixel-by-pixel basis. In one preferred embodiment, the 
image is segmented into regions or image segments, and in another embodiment the 
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image areas are analyzed to identify pixels according to a hierarchy, such as blobs and 
sub-blobs. In another embodiment, the image areas are analyzed to identify edge 
portions separate from other areas of the image, for example to allow edge 
enhancement, and in another embodiment, the images are analyzed to identify interior 
regions of an image that form a part of a larger part of the image. According to one 
embodiment, each pixel is characterized as either white, black or color using an 
appropriate standard, and in another embodiment, each pixel is characterized as either 
white, white edge, gray, gray edge, black, color or color edge. In one embodiment, the 
image analysis and characterization are carried out with one purpose being the 
identification of text-only images. In other situations, the analysis and characterization 
can be carried out in order to enhance the image, to compress the image or to segment 
the image for further processing. 

The pixel image data is also analyzed to identify appropriate relationships 
between pixels, especially those pixels adjacent to each other. In one preferred 
embodiment, pixels that are adjacent each other and that have the same 
characterization, such as black, white or color, are then grouped, aggregated or linked 
to each other and identified as connected regions, sometimes referred to as blobs. In 
the preferred embodiments, the characterization of parts of the image as being related 
is intended to recognize and take advantage of the fact that related parts of an image 
will have identical or similar characteristics, or at least sufficiently similar to allow those 
parts of the image to be treated together, either in terms of enhancement, segmentation 
or separation or for other purposes. For example, regions of all black text will share the 
characteristics that the pixels will be either black, white or relatively sharp or steep 
transitions between those two extremes, and black regions and white regions will be 
relatively uniform in intensity or luminance. On the other hand, pictorial or graphic 
regions, represented in gray scale, will have substantially more gray areas, softer or 
smoother transitions between light and dark and comparatively fewer aggregations of 
black and white pixels. Therefore, in the preferred embodiments, the pixels are not only 
analyzed for their content but also their relationship to adjacent and nearby pixels. 

In one example, the image data is then processed line-by-line, and preferably on 
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a pixel-by-pixel basis, to analyze and characterize each pixel, and, if possible, to group 
the pixels. The image data is checked for background levels so that digital values 
corresponding to true white and true black can be established. Digital values in 
between will be assigned different gray scale values, and gray values may be assigned 
using a threshold determination, such as anything between 0.5 and 0.95 times the value 
of the white pixels. Black may then be anything equal to or below 0.5. 

The CPU uses a gradient operator 1 16 to determine if a given pixel represents or 
is immediately adjacent an edge. The processor checks to see if the data is going from 
light to dark or dark to light and provides a representation as to the amount of change 
and the direction of change. The direction of the change will indicate whether the data 
is going from light to dark or from dark to light. The processor then determines whether 
or not the change is significant or large enough to represent an edge or is slight enough 
to represent the more gradual changes present in photo or graphic images. 

The threshold or thresholds established by the processor and the gradient values 
corresponding to the pixels in the line being analyzed are then used by the processor to 
characterize 118 each pixel. Using the threshold and the indication of whether or not 
the pixel is on an edge, the processor decides whether each pixel should be 
characterized as white, black or gray or color. If the pixel is above the threshold, it will 
be characterized as white, and if it is at a digital value at the other end of the spectrum 
(below the gray threshold), it will be characterized as black. If it is in between, and the 
gradient value is small (below the gradient threshold, such as below 30) or otherwise 
indicating a gradual change, the pixel is characterized as gray scale or color. The 
characterization of each pixel is then stored, along with the gradient value, if desired. 
Additionally, each pixel can be assigned one of more than three values, in other words 
intermediate values in addition to black, white or gray or color. For example, possible 
labels for each pixel may include dark gray, light gray, black, white, color, gray edge, 
color edge or other useful characterizations. The labels or characterizations and their 
number may depend on what the characterizations ultimately will be used for and the 
desired quantity and precision desired for the data. 

Preferably after the pixel classifier, the processor analyzes 120 the connectivity 
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or relationships among pixels to identify connected regions or areas of pixels that can 
be characterized the same or sufficiently similar to represent one type of pixel. For 
example, if one or more pixels immediately adjacent a given pixel has the same label, or 
a label that is sufficiently close to be characterized as the equivalent, those pixels will be 
grouped, separated or otherwise combined into one region type or other 
characterization. For example, all adjacent pixels labeled black may ultimately be 
identified as text, especially if there are a significant number of adjacent pixels that are 
labeled white. However, if those pixels labeled black are in or surrounded by regions of 
different gray scale levels, those pixels as well as the gray scale level pixels may be 
identified as a graphic or pictorial region, and possibly color text. 

With the information from the connectivity analysis, the processor can then 
characterize 122 each region or blob. The processor can then tabulate, sum or 
otherwise account for the relative numbers of text, graphic or other characterizations of 
regions. The image can be processed or stored, as desired. For example, where the 
application or destination of the image is known, the processing of the image can be 
completed. Where the image is to be enhanced, the text regions can be edge 
sharpened and the text characters made all black. The color regions can be smoothed 
and some color manipulation can also be done, if desired. The text and graphic regions 
can be compressed as well. 

In one example of image analysis, the process uses three sequential steps, 
which will be called pixel classification, hierarchical connected component (HCC) 
analysis and region or blob classification. Pixel classification preferably identifies or 
uses image features to distinguish the different parts of the image. In example where 
the image data is analyzed in gray scale form, those features are representations of 
black and white values, edge elements, and the balance, which will be called gray. In 
an example where the image includes color data, the features used to characterize 
each pixel may be black, white, white edge, gray, gray edge, color and color edge. 
Other features or image characteristics may be used to differentiate parts of the image, 
but it is believed that these are particularly appropriate for many of the images that 
would be encountered in ordinary practice. Hierarchical connected component (HCC) 
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analysis and region or blob classification are described in Applicant's specification. 
Examples of the applications of these processes are also provided in Applicant's 
specification. 

Cited Prior Art 

All the references relied upon in the Office Action are strictly binary processes. 
The pixels in the image are converted to binary either black or white. Hussein teaches 
an image pre-processor for character image recognition. The system scans a 
document and obtains a gray-scale image of the document. The system then generates 
a binary image and compares the gray-scale image with a threshold to segment the 
binary image into individual characters and to determine the shape of the individual 
characters. Gray-scale image information is extracted from the gray-scale image for 
each individual character based on the location and shape of the character in the binary 
image, and the image information is recognized to determine the identity of the 
character. Because the system operates on a binary image, the data representing the 
gray scale image is necessarily characterized as either black or white, and no other 
image characteristics are identified with pixels. The binary image is used to move under 
lines and to obtain connected components within a binary image. The system also uses 
the binary image to separate text-type components from non-text type components so 
that only text-type connected components can be processed for character recognition. 
As acknowledged in the Office Action, Hussein does not disclose identification of an 
element having a given characteristic that is adjacent another element having a 
characteristic approximately the same as the given characteristic. However, the 
secondary reference cited in the Office Action does not provide the missing subject 
matter. 

The Office Action cites Luther et al. as the secondary reference, but Luther et al. 
does not teach or suggest claim elements such as identifying edge information, color 
information, gray scale information, or applying sub-labels to interconnected pixels. 
Specifically, Luther et al. teaches a method for capturing a document image and a 
document image management system. In the system, a scan provides by-level image 
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data analyzed to identify blocks of uniform image type, for example text, line drawing, 
gray scale image, or full-color image within the document. While the Office Action 
argues that Luther identifies each element having a given characteristic approximately 
the same as a given characteristic of an adjacent element, nothing in Luther or in Al- 
Hussein teaches or suggests such methods as detecting an edge using a gradient to 
detect an edge, detecting gray, and gray edge, or detecting white edge, gray edge, and 
color edge, to cite some examples. 

Wang et al., U.S. Patent No. 5,680,479, teaches contour tracing, in conjunction 
with FIG. 4, at column 11, line 25, through column 12, line 27, and following. This 
patent is cited in Luther at column 10, line 37, column 14, line 60, column 16, line 57, 
and column 1 9, line 12. However, Wang et al. operates only on binary data. [See, 
column 8, lines 40-43 ("convert the non-binary pixel data into binary pixel data prior to 
storage in RAM 20").] Using the binary pixel data, the contour tracing does not find 
elements representing an edge but instead operates on binarized black pixels to 
develop a contour. It appears from Wang et al., that on average, half of the pixels 
would be made black and half would be made white for those pixels in Applicant's 
process that would be identified as edge elements. 

It is noted also that Wang etal. also defines a rectangle around each connected 
component, which has no relation to any actual edge configuration of a connected 
component. [See, column 1 1 , lines 66-67.] It is also clear that Wang does not locate 
edges, because no interior portions of a component is traced. [See, column 12, lines 
10-12, "interior portions of closed contours are not traced."] Therefore, Wang et al. 
would not even find an interior opening or edge thereof for the letter "A" or the letter "Q" 
as depicted in FIG. 5(a) of Wang. 

As can be seen, all the references in the Office Action convert the pixels in the 
image to binary either black or white. No edge detection occurs. See, Al-Hussein, 
column 12, lines 45-51, and column 18, lines 9-13. See, Luther, column 7, lines 45-48 
("black and white (bi-level)", and column 10, line 38 and column 14, line 60 referring to 
the Wang etal. patent. See, also the Wang etal. patent, column 8, lines 40-43 
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("convert the non-binary pixel data into binary pixel data prior to storage in RAM 20"). 
Clearly, Applicant has taught inventions patentable over the references. 

Claims 

Consider now the claims in the application. Claim 1 is an independent method 
claim and recites in part: 

"receiving data representing a plurality of elements of an image; 

"characterizing each element in the plurality of elements according to a 
perceived characteristic corresponding to an edge; and 

"identifying each element having a given characteristic that is adjacent an 
element having a characteristic approximately the same as the given 
characteristic." 

None of the cited references taken singly or in combination teach or suggest the 
claimed combination, the recited elements quoted above, or "characterizing each 
element in the plurality of elements according to a perceived characteristic including a 
characteristic_corresponding to an edge". The cited references, singly or in 
combination, teach and suggest nothing about characterizing an element characteristic 
corresponding to an edge. The cited references are black and white pixel 
classifications, in a binary fashion. Clearly, claim 1 is patentable over the art. 

The claims 2-21 are dependent directly or indirectly from independent claim 1 
and are asserted as being patentable for the same reasons as discussed above with 
respect to claim 1 , for the additional combinations in the dependent claims as well as for 
the additional limitations recited in the dependent claims. Note for example, claim 6 
reciting in part "wherein the step of characterizing includes the step of identifying pixels 
representing color information". Note also claim 18 reciting in part "wherein the step of 
identifying each element that is adjacent includes the step of identifying adjacent pixels 
that are characterized as background, and further including the step of identifying 
adjacent pixels characterized as background and also characterized with a label". 
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Claim 19 recites in part "wherein the step of identifying each element that is adjacent 
includes the step of identifying adjacent pixels that are characterized as non- 
background, and further including the step of identifying adjacent pixels characterized 
as non-background and also characterized with a label". 

Claim 22 is an independent method claim and recites in part: 

"identifying elements of the plurality of elements of the image that 
represent an edge of a portion of the image; and 

"identifying elements of the image that represent an edge and that 
are adjacent at least one other element representing an edge." 

None of the cited references taken singly or in combination teach or suggest the 
claimed combination, the recited elements quoted above, or "identifying elements of the 
plurality of elements of the image that represent an edge of a portion of the image" or 
"identifying elements of the image that represent an edge and that are adjacent at least 
one other element representing an edge". 

The claims 23-43 are dependent directly or indirectly from independent claim 22 
and are asserted as being patentable for the same reasons as discussed above with 
respect to claim 22, for the additional combinations in the dependent claims as well as 
for the additional limitations recited in the dependent claims. Note for example claim 23 
reciting in part "wherein the step of identifying elements of the image representing an 
edge includes the step of using a gradient operation on groups of pixels to determine if 
an individual pixel is part of an edge". Claim 24 recites in part "wherein the step of 
identifying elements of the image that represent an edge and that are adjacent includes 
the step of selecting a first pixel that represents an edge, and identifying a plurality of 
adjacent pixels, and identifying any of the plurality of adjacent pixels that represent an 
edge". Claim 27 recites in part "wherein the step of identifying any of the plurality of 
adjacent pixels that represent an edge includes the step of labeling the adjacent pixels 
that represent an edge with a unique label". Note also claim 33 reciting in part "wherein 
the step of identifying pixels having the same label that are connected includes the step 
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of applying a sub-label to the connected pixels that have the same label". Claim 43 
recites in part "further comprising the step of counting the number of non-black pixels 
that are interconnected and comparing to the number of interconnected black pixels". 
Claim 44 is an independent method claim and recites in part: 

"identifying elements of the plurality of elements of the image that 
represent an edge of a portion of the image; and 

"identifying elements of the image that represent an edge and that 
are adjacent at least one other element representing an edge." 

None of the cited references taken singly or in combination teach or suggest the 
claimed combination, the recited elements quoted above, or "identifying elements of the 
plurality of elements of the image that represent an edge of a portion of the image" or 
"identifying elements of the image that represent an edge and that are adjacent at least 
one other element representing an edge". 

The claims 45-49 are dependent directly or indirectly from independent claim 44 
and are asserted as being patentable for the same reasons as discussed above with 
respect to claim 44, for the additional combinations in the dependent claims as well as 
for the additional limitations recited in the dependent claims. Note for example claim 46 
reciting in part "further including identifying elements of the plurality of elements of the 
image that represent black, gray, and gray edge". Claim 47 recites "wherein identifying 
elements of the plurality of elements of the image that represent an edge of a portion of 
the image include identifying elements that represent one of a white edge, gray edge, 
and color edge". Claim 49 recites "wherein identifying elements of the image that 
represent an edge includes identifying elements of an image representing an edge 
using a gradient operation". 

Claim 50 is an independent method claim and recites in part: 
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"characterizing, as a function of the grayscale image data, elements 
in the plurality of elements according to a perceived characteristic wherein 
a perceived characteristic includes an edge characteristic 

None of the cited references taken singly or in combination teach or suggest the 
claimed combination, the recited elements quoted above, or "characterizing, as a 
function of the grayscale image data, elements in the plurality of elements according to 
a perceived characteristic wherein a perceived characteristic includes an edge 
characteristic". 

The claims 50-57 are dependent directly or indirectly from independent claim 50 
and are asserted as being patentable for the same reasons as discussed above with 
respect to claim 50, for the additional combinations in the dependent claims as well as 
for the additional limitations recited in the dependent claims. 

Reconsideration of the application and claims in view of the foregoing 
amendments and remarks is respectfully requested. Early notice of allowance thereof is 
earnestly solicited. 

This response is being filed with an Information Disclosure Statement and a One- 
Month Extension of Time. 

If a petition is required in conjunction with this paper, please consider this a 
request for such a petition. 

Respectfully submitted, 

Dated: March 21 , 2007 /James A. Henricks/ 

James A. Henricks 
Registration No. 31,168 
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